Skip to content

target: track explicit test cache counts#11948

Open
sluongng wants to merge 2 commits into
masterfrom
sluongng/target-cache-counts
Open

target: track explicit test cache counts#11948
sluongng wants to merge 2 commits into
masterfrom
sluongng/target-cache-counts

Conversation

@sluongng

Copy link
Copy Markdown
Contributor

A user reported that the test grid could show cached tests as
non-cached. That happened because test target history inferred cache
state from timestamps and flattened multiple test attempts down to a
lossy bool.

This change keeps target history test-only, adds explicit local,
remote, and total cached attempt counts to ClickHouse-backed test
history, and updates the TAP grid to derive full versus partial
cache state from those fields. The primary DB path stays on the legacy
bool, and cached remains in TargetStatus as a fallback for older
ClickHouse rows that do not have the new count columns yet.

This requires an additive ClickHouse migration for TestTargetStatuses.
Older rows still render via the legacy bool and timestamp heuristic
until they are rewritten, while new rows carry the full cache-attempt
semantics.

@sluongng sluongng requested review from bduffany and luluz66 April 22, 2026 14:41
@sluongng sluongng marked this pull request as ready for review April 22, 2026 14:41
TargetType int32
TestSize int32
Status int32
Cached bool

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

want to mark this as // Deprecated? (I guess we probably can stop writing this value eventually since it can be derived from the other columns?)

Comment thread proto/target.proto Outdated
// The number of cached test attempts served from the remote cache.
int32 cached_remotely_count = 9;

// The total number of recorded test attempts for this target invocation.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this include cached attempts? (maybe clarify in the comment)

Comment thread proto/target.proto
Comment on lines +58 to +61
// Whether all recorded test attempts for this target invocation were served
// from cache. Older ClickHouse rows may only populate this legacy field
// without the explicit cache counts below.
bool cached = 6;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without the explicit cache counts

Is this equivalent to checking total_run_count == 0? (maybe be more explicit in this comment)


func (t *target) effectiveTotalRunCount() int32 {
count := t.totalRunCount
if t.observedResultCount > count {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we expect this to ever happen? if not, maybe add a debug log here (with CtxDebug)?

@sluongng sluongng force-pushed the sluongng/target-cache-counts branch 3 times, most recently from b7faa3c to 6535b95 Compare May 18, 2026 14:29
sluongng added 2 commits June 24, 2026 11:27
A user reported that the test grid could show cached tests as
non-cached. That happened because test target history inferred cache
state from timestamps and flattened multiple test attempts down to a
lossy bool.

This change keeps target history test-only, adds explicit local,
remote, and total cached attempt counts to ClickHouse-backed test
history, and updates the TAP grid to derive full versus partial
cache state from those fields. The primary DB path stays on the legacy
bool, and cached remains in TargetStatus as a fallback for older
ClickHouse rows that do not have the new count columns yet.

This requires an additive ClickHouse migration for TestTargetStatuses.
Older rows still render via the legacy bool and timestamp heuristic
until they are rewritten, while new rows carry the full cache-attempt
semantics.
Address review feedback about ambiguity around when the legacy cached
bool should be trusted because older rows do not have explicit count
columns. Spell out that total_run_count == 0 is the legacy no-counts
signal, mark the stored cached column as deprecated for new rows, and
log when observed TestResult counts override summary counts.
@sluongng sluongng force-pushed the sluongng/target-cache-counts branch from 6535b95 to cc5c124 Compare June 24, 2026 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants